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ABSTRACT 





The Training & Doctrine Command (TRADOC) Analysis Center (TRAC) — 
Monterey, California re-engineered the Janus simulation as a technology demonstration. 
The completed simulation, HLA Warrior, applied modern technologies including an 
object-oriented design and state-of-art user interfaces. The project also re-wrote Janus 
source code in C++. The purpose of this thesis was to assess HLA Warrior’s fidelity, 
defined as its ability to replicate Janus results, by conducting a statistical comparison of 
Janus and HLA Warrior. Given that Janus has high “face-validity,” Janus results acted as 
the baseline from which HLA Warrior results were compared. The comparison involved 
executing identical scenarios in Janus and HLA Warrior, gathering results, and 
conducting a rigorous statistical comparison of Janus and HLA Warrior results. 
Statistical tests included the paired t-test and non-parametric Wilcoxon Signed Ranks 
Test. 

Results from the tests showed differences between Janus and HLA Warrior. 
Investigation into the causes of the differences found two source code errors in HLA 
Warrior. Re-evaluation of HLA Warrior following correction of the errors resulted in a 
reduction in magnitude of the differences. Probable causes due to algorithm 
implementation differences were also identified. While differences exist, HLA Warrior 


appears to have face-validity and generally produces outcomes similar to Janus. 
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EXECUTIVE SUMMARY 


The United States Army’s Training and Doctrine Command Analysis Center- 
Monterey (TRAC-Monterey) recently re-engineered the Janus simulation as a technology 
demonstration. Technologies included an innovative architecture, state-of-art user 
interfaces, and built-in High-Level Architecture (HLA) tools. The project ported Janus to 
a personal computer (PC) running the Windows NT (WinNT) operating system and 
rewrote J — source code in C++ while maintaining the integrity of the Janus 
algorithms. The resulting simulation is known as HLA Warrior. 

The purpose of this thesis was to assess HLA Warrior’s fidelity, defined as the 
simulation’s ability to replicate Janus results, by conducting a statistical comparison of 
Janus and HLA Warrior scenarios. The research determined whether results from J anus 
and HLA Warrior scenarios were statistically similar. Since Janus has high “face- 
validity,” it acted as the baseline to compare HLA Warrior’s results. A model with high 
face validity is defined as a model that, on the surface, seems reasonable to people who 
are knowledgeable about the system under study [Ref. 1:p. 308]. 

Janus is a high-resolution, ground combat simulation focusing on maneuver and 
artillery units. Janus represents entities down to individual systems or soldiers. Janus 
stochastically adjudicates all detections and engagements between individual systems. 
Janus contains a post-processor that permits the analyst to gather comprehensive statistics 
on the simulation run, including reports on the number of rounds fired, detection ranges, 
kill ranges, Force Exchange Ratio (FER), and Loss Exchange Ratios (LER). 

To assess whether HLA Warrior can replicate Janus scenario results, the research 


analyzed measures of performance (MOPs) that quantified the engagement process. The 


XI 





fundamental steps of an engagement sequence include detecting then shooting an entity. 
Selected MOPs included detection range, kill range, and number of rounds fired. The 
research also analyzed force exchange ratio (FER) as a quantitative measure of overall 
battle outcome. 

Two scenarios were developed to analyze results from varying weapons systems: 
a mechanized scenario and a light infantry scenario. Each scenario was then executed in 
two environments, Fort Hunter-Liggett, California and Southwest Asia. These distinctly 
different environments support the analysis as to whether HLA Warrior successfully 
models the engagement process under varying line-of-sight conditions. 

Each of the two scenarios (mechanized, light) was executed in two environments 
(Southwest Asia, Fort Hunter-Liggett) and run in two different modes (Janus, HLA 
Warrior). The two scenarios, two environments, and two modes resulted in a total of 
eight combinations for the experiment. Each of the combinations was executed 10 times. 
In total, 80 runs were performed to provide a reasonable data sample to analyze each 
MOP. 

The method of analysis was to compare the MOPs resulting from a specific 
number of Janus runs to the MOPs resulting from the same number of HLA Warrior runs. 
Since Janus produces exactly the same results when the same random number seed is 
used (likewise for HLA Warrior), all Janus runs were executed using different, randomly 
selected seeds. The same random number seeds were then used in the corresponding 
HLA Warrior runs. The combination of the same scenario, same environment, and 
Same random number seed executed in opposing modes (Janus and HLA Warrior) 
support paired output from the two simulations. 


Xil 











After completing all runs and gathering required data, MOPs were analyzed using 
the Wilcoxon Signed Ranks Test. Then those MOPs that satisfied the paired t-test’s 
normality assumption as determined by observation of normal probability plots were 
analyzed using the more powerful paired f-test. 

The results of the analysis showed a disparity between HLA Warrior and Janus. 
Further investigation into the causes of the disparity led to two possible explanations. 
First, two errors in the HLA Warrict source code were found. Re-evaluation of HLA 
Warrior following correction of the source code errors resulted in a reduction in the 
magnitude of the differences between Janus and HLA Warrior. A second possible cause 
of the differences between the simulations was a difference in the algorithm 
implementation methodology between Janus and HLA Warrior. While the integrity of 
the Janus algorithms were maintained in HLA Warrior, developers implemented 
algorithms differently in HLA Warrior to improve efficiency. The implementation 
differences could lead to a disparity in the results of the investigated MOPs. 

While the two simulations are different with regards to selected MOPs, HLA 
Warrior generally produces similar outcomes to Janus. In general, the flow of the battles 
executed in HLA Warrior progressed as a subject matter expert might expect, lending a 


degree of face-validity to HLA Warrior. 
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I. INTRODUCTION 


The United States Army’s Training and Doctrine Command (TRADOC) Analysis 
Center-Monterey, California (TRAC-Monterey) recently re-engineered the Janus 
simulation with modern technologies, including an innovative architecture, state-of-art 
user interfaces, and built-in High-Level Architecture (HLA) tools. The project ported 
Janus to a personal computer (PC) running the Windows NT (WinNT) operating system 

and rewrote Janus source code in C++. The resulting simulation is known as HLA 
Warrior. HLA Warrior’s primary objective was to serve as a technology demonstration, 
providing lessons learned and reusable products to developers of future simulations. 

The purpose of this thesis was to assess HLA Warrior’s fidelity, defined as the 
simulation’s ability to replicate Janus results, by conducting a statistical comparison of 
Janus and HLA Warrior scenarios. The research determined whether results from Janus 
and HLA Warrior scenarios were statistically similar. Since Janus has high “face- 
validity,” it acted as a baseline to compare HLA Warrior’s results. A model with high 
face validity is defined as a model that, on the surface, ‘ine reasonable to people who 
are knowledgeable about the system under study [Ref. 1:p. 308]. Subject matter experts 
(infantry officers, armor officers, and so on) have examined the execution of various 
Janus scenarios and determined that the way in which the battles progressed and the 
outcomes of the battles were plausible and likely replicated real-world phenomena. 

In order to test for similarity between Janus and HLA Warrior, measures of 


performance (MOPs) were developed. Where the simulation results were statistically 











similar, the findings aided in the validation of HLA Warrior. The thesis also identified 
factors that may account for any significant differences between the two simulations. 
A. JANUS 

Lawrence Livermore National Laboratory originally developed Janus in the 
1970s, with the current version developed by TRAC-White Sands Missile Range, New 
Mexico. Most Janus systems use a UNIX operating system and execute on Hewlett- 
Packard workstations. 

Janus is a ground combat simulation primarily focusing on maneuver and artillery 
units. Janus also models engineer support, minefield employment and breaching, rotary 
and fixed-wing aircraft, chemical environments, limited weather effects, and day and 
night visibility. Janus is a high-resolution simulation, representing entities down to — 
individual systems or soldiers. The user may aggregate entities when appropriate. 

[Ref. 2:p. 21] 

Analysts use Janus to conduct in-depth studies of tactics, techniques, and 
procedures. Analysts can also use Janus to evaluate new weapon systems in various 
environments, weather conditions, and mission profiles. Janus is also used to train 
brigade and battalion staffs. 

Janus is interactive in that interplay between operators during simulated combat is 
possible. Users may influence scenario results by altering movement routes, planning 
and firing artillery missions, and mounting/dismounting entities on vehicles. User 
interaction is not necessary for pre-planned scenarios. For purposes of this thesis, all 


scenarios were pre-planned and included no user interaction during execution. 

















Janus stochastically adjudicates all detections and engagements between 


individual systems. Prior to the Janus run, the user may specify a random number seed 

up to seven digits, allow the system to randomly specify a seed, or use the default random 
number seed hard-coded into the program [Ref. 2:p. 42]. When the same random number 
seed is used in the same scenario without human interaction, the exact same results occur. 

Janus possesses a robust database that permits the user to define a weapon system 
extensively and capture detailed factors required for scenario development. Individual 
fighting systems have distinct properties: dimensions, weight, carrying capacity, and 
speed. Users can also modify weapons system parameters such as range, ordnance type, 
and ammunition basic load (supply of ammunition each entity carries into a battle) 

[Ref. 2:p. 4]. 

Janus’ post-processor permits analysts to gather comprehensive statistics on the 
simulation run, including reports on the detection ranges, kill ranges, chemical casualties, 
Force Exchange Ratio (FER), number of rounds fired, and Loss Exchange Ratios (LER). 
B. HLA WARRIOR 

TRAC-Monterey re-engineered the Janus simulation with several modern 
technologies, including an innovative architecture, windows-like user interfaces, and 
HLA tools. The project ported Janus to a PC runnin g the WinNT operating system and 
rewrote Janus source code in C++ while maintaining the integrity of the Janus 
algorithms. HLA Warrior implements new graphical user interfaces (GUIs) using Vision 
AX] management tools that comply with Distributed Interactive Simulation (DIS) and 


HLA requirements. The U.S. Army National Simulation Center (NSC) will also expand 








HLA Warrior capability by integrating Operations Other Than War (OOTW) algorithms 
from the Spectrum simulation. [Ref. 3:p. 1]. 

HLA Warrior possesses all of the modeling and analytical capabilities of Janus. 
HLA Warrior also has the capability to read and execute scenarios developed in Janus 
including entity locations and movement routes. This function proved useful in the 
design of the experiment because once a scenario is developed in Janus, its initial settings 


can be duplicated in HLA Warrior prior to the run. 











Il. PROBLEM DESCRIPTION 


The purpose of this thesis was to assess HLA Warrior’s fidelity, defined as the 
simulation’s ability to replicate Janus results, by conducting a statistical comparison of 
Janus i HLA Warrior scenarios. To answer this question, the thesis tested HLA 
Warrior’s ability to accurately replicate specific combat functions under varying 
conditions. Of primary concern is the ability of HLA Warrior to replicate the basic 
engagement process, detecting and killing a target. 

A primary element of detecting and subsequent killing of a target in ground 
combat is line of sight. The target must be in the line of sight of the shooter in order to be 
detected. Further, the target must remain within line of sight of a direct fire weapon 
throughout the engagement process to be killed. Therefore, HLA Warrior must 
accurately replicate Janus’ line-of-sight calculations. | 

Provided the line-of-sight calculations are accurate, HLA Warrior must replicate 
Janus’ detection and kill algorithms accurately in order for HLA Warrior to be similar to 
Janus. Both Janus and HLA Warrior use a stochastic process to adjudicate detections and 
kills. This implies that both the random number generators and the algorithms that 
assess the probabilities of hit and kill in the two simulations need to be similar. 

This thesis therefore provided a method to assess HLA Warrior’s line-of-sight 
algorithms, random number generators, detection algorithms, and kill algorithms for 


comparison to Janus. 





The following chapter describes the design of the experiment to determine 
_ whether Janus and HLA Warrior are statistically similar and address the experimental 


issues raised above. 














Hit. DESIGN OF EXPERIMENT 


The goal of the experiment was to provide a means to obtain data from identical 
scenarios executed in both Janus and HLA Warrior for the purpose of analysis. Design 
issues included selecting appropriate and sufficient data to be analyzed, determining the 
nature of the scenarios, and conducting the experiment. 

A. MEASURES OF PERFORMANCE 

The first issue in the design of the experiment was to identify and select data to be 
analyzed. The Measures of Performance (MOPs) were limited to information available 
from the simulations’ post-processors. The MOPs also had to support the analysis of the 
issues raised in the Problem Description. To assess whether HLA Warrior can replicate 
Janus scenario results, the research analyzed MOPs that quantified the engagement 
process. The fundamental steps of an engagement sequence include detecting then 
shooting an entity. Selected MOPs included detection range, kill range, and number of 
rounds fired. The research also analyzed force exchange ratio (FER) as a quantitative 
measure of overall battle outcome. | 

1. Detection Range 

Detecting a target is the first basic element of a battle. Before a combat system 
can engage and subsequently kill a target, the target must first be detected. Comparing 
the detection ranges produced by Janus and HLA Warrior is a quantitative method of 
assessing the line-of-sight calculations as well as the detection algorithms in HLA 


Warrior. 


2. Kill Range 

The second basic element in a battle is killing a target once detected. Analyzing 
the kill ranges of Janus and HLA Warrior provides evaluation of the line-of-sight 
algorithms and a quantitative method for evaluating HLA Warrior’s engagement process. 

3: Force Exchange Ratio 

Generally, a combat simulation’s overall objective is to accurately determine the 
victor in a given engagement. The force exchange ratio, defined by the equation below, 

(Rediosses / Bluejosses) / (REdinitiat / Blue initiat) 

is a quantitative measure of the scenario outcome. In words, the FER is the quantity of 
the loss exchange ratio divided by the initial force ratio. The FER is a more accurate 
indicator of the outcome than the loss exchange ratio (total red side losses divided by 
total blue side losses) because it fle: initial force levels into account and standardizes 
losses. That is, if two opposing forces lose an equal number of systems, losses to the 
smaller force are more damaging than losses to the larger force. 

4. Rounds Fired 

_ Not all shots result in kills. Therefore, analysis of the total number of rounds 

fired in both models is useful to assess the engagement process. Comparing the total 
number of rounds fired in both simulations can also support analysis of Janus’ kill 
algorithms and probability of kill processes being replicated in HLA Warrior. 
B. SCENARIOS 

The scenarios were developed based on the requirements posed in the Problem 


Description. It is important to analyze results from varied scenarios to model varying 











weapons systems and their capabilities. Therefore the experiment included scenarios 
using dissimilar systems, an armored/mechanized scenario and a light infantry scenario. 

1. Mechanized Forces 

The first scenario was an armored/mechanized force scenario. The scenario 
consisted of a tank company reinforced with armored personnel carriers defending and a 
similarly equipped battalion attacking. Both sides were armed with M1A1 Abrams main 
battle tanks and M2 Bradley armored personnel carriers (APCs). The defenders had 14 
tanks and four APCs. The attacking force consisted of 39 tanks and eight APCs. The 
initial force ratio of attacker to defender was approximately 3:1 for armored vehicles. 
Additionally, both sides were equipped with two AH-64 Apache helicopters as well as air 
defense systems. 

Zs Light Forces 

The second scenario was a light infantry scenario consisting of a rifle company 
defending and a rifle battalion attacking. The defending company consisted of 103 
entities comprised of four system types including riflemen, light machine guns, machine 
guns, and light anti-tank weapons (LAW). The attacking battalion was similarly 
equipped with a total of 285 entities, resultin g in an initial force ratio of approximately 
3:1. 
C. ENVIRONMENTS 

_ In order to assess HLA Warrior’s detection and kill algorithms, the selected 

scenarios were executed in contrasting environments. A hilly, wooded environment and 


a flat, desert environment were selected for the experiment. These distinct environments 





support the analysis as to whether HLA Warrior accurately models the detection and kill 
algorithms under varying line-of-sight conditions. 

1. Fort Hunter-Liggett (HL), California 

The Fort Hunter-Liggett terrain file provided a hilly, wooded environment. The 
hills and trees test the line-of-sight calculations and, consequently, the detection and kill 
algorithms in a constrained visual environment. The hills, when combined with the 
effects of the trees, serve to limit line of sight and adversely effect the detection and kill 
process. 

2. Southwest Asia (SWA) 

The Southwest Asia terrain file provided a relatively flat, featureless, desert 
environment. The terrain is less restrictive to line-of-sight than Hunter-Liggett terrain. 
Therefore the detection and kill ranges should approach maximum effective range of 
individual sensors and weapon systems. 

D. MODELS 

After determining the scenarios and environments, four separate models were 
developed: one mechanized and one light scenario in Southwest Asia and one 
mechanized and one light infantry scenario at Fort Hunter-Liggett. Modification of the 
Janus weapons system database ensured that each weapon system on one side of the 
battle could engage each type of system on the opposing side. 

In each scenario, the blue force (the defending force) was positioned on easily 
defendable terrain. The fields of view vary from entity to entity to adequately test the 
line-of-sight calculations. All blue force ground entities (tanks, APCs, infantrymen, etc.) 
remained stationary throughout the battle. The blue AH-64s in the mechanized scenarios 
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traveled along pre-planned routes. Both AH-64s began movement as the simulation 
commenced. 

Fach entity in the red force (the attacking force) traveled along a pre-planned 
movement route toward an objective occupied by the blue force. Additionally, the red 
force AH-64s in the mechanized scenarios traveled along a route that took them over the 
blue force. All red force movement commenced as the simulation began. 

Although one of the purposes behind the development of HLA Warrior was to 
ease the scenario building process, no scenarios were constructed in HLA Warrior. 
Rather, HLA Warrior’s ability to read and convert Janus scenario files was utilized. This 
served to ensure that the four models executed in HLA Warrior had Starting positions, 
routes, and pre-planned missions identical to the four models executed in Janus. 

E. EXPERIMENT 

1. Conduct of Experiment 

Each of the two scenarios (mechanized, light) was executed in two environments 
(Southwest Asia, Fort Hunter-Liggett) and run in two different modes (Janus, HLA 
Warrior). The two scenarios, two environments, and two modes resulted in a total of 
eight combinations for the experiment. Each of the combinations was executed 10 times. 
In total, 80 runs were performed to provide a reasonable data sample to analyze each 
MOP. 

The method of analysis was to compare the MOPs resulting from a specific 
number of Janus runs to the MOPs resulting from the same number of HLA Warrior runs. 
Since Janus produces exactly the same results when the same random number seed is 
used (likewise for HLA Warrior), all Janus runs were executed using different randomly 
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selected seeds. The same random number seeds were then used in the corresponding 
HLA Warrior runs. While the random number generator in HLA Warrior is intended to 
be identical to the random number generator in Janus, the implementation of the 
algorithms (discussed in chapter 4, Statistical Analysis), caused the random number 
draws in the two simulations to lose synchronization rapidly. Despite the differences in 
the random number utilization, the design of the experiment supports paired output from 
the two simulations. 

2. Experimental Design Issues 

Determining sample size is fundamental to the design of every experiment. An 
experiment is conducted a number of times so that the data produces good estimators of 
the true population parameters. Several techniques are available to arrive at a satisfactory 
sample size. 

First, experimenters may apply a practical approach. Based on experience and 
recommendations from senior Janus analysts, a sample size of n = 10 generally produces 
results with “acceptable variance” for the defined MOPs. - That is, the experiment is run 
until an estimate of the variance for the mean is reduced to a pre-determined, acceptable 
level. The acceptable level will be different for each MOP. 

The issue of normality of data was also considered when determining sample size. 
Normality of the data set is one assumption required prior to applying the paired r-test 
(described below). Normal probability plots were used to test for approximate normality. 
As the sample size increases, one would expect the sample averages to become more 


normal, based on the Central Limit Theorem [Ref. 4:p. 232]. Based on observations of 
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normal probability plots for each MOP, a sample size of n = 10 was generally considered 
large enough to produce normally distributed averages. 
F. DATA COLLECTION 

Data was collected following the 10 runs of each of the eight experimental 
designs described above. Summary statistics for each MOP were computed based on the 
raw data for each run (see Appendix A). Each cell entry for detection and kill range in 
Appendix A represent an average of all detection ranges and kill ranges for the specific 
scenario, environment and mode for a specific scenario run. The entries for Force 
Exchange Ratio and Rounds Fired are the final force exchange ratios and the actual 
number of rounds fired for each specific scenario run respectively. Tables 1-3 below 


present the summary statistics used in the analysis of each MOP. 














Summary Detection Range Blue =e Range Red | 
Statistic | Warrior | Janus __ 


| Janus | 
Standard Deviation 8, 882 28. 961 20. 415 9, 096 
Standard Deviation 47. 51 9 221. 948 91 092 200. 412 
Standard Deviation 47. 685 45. 207 36. 954 26. 454 
Standard Deviation 314. 325 116. 839 201 431 228. 863 


Table 1. Detection Range Summary Statistics 
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Summary 

Pa Se ee ee 
Standard Deviation 43.028 47.663 36.116 25.195 

Standard Deviation 126.787 286.633 109.508 258.894 

Standard Deviation] 190.953 32.414 70.824 41.741] © 

Standard Deviation} 290.446 194.556 212.336 286.861 


Table 2. Kill Range Summary Statistics 







Statistic | Warrior | Janus | __Warrior | Janus 


HL Liaht Average 0.187 0:171 6690.200 71883.100 
y Standard Deviation 0.031 0.056 425.022 4316.057 


pany Summary | FER | Rounds Fired 


HL Mech Average 0.793 1.082 323.800 511.500 
Standard Deviation 0.143 0.480 33.963 180.981 

SWA Light Average 0.746 0.326!  18373.900 27723.300 
g™ | Standard Deviation 0.225 0.089]  1518.918 4126.873 
Average 0.128 1.770 46.700 291.900 

Se Mcen Standard Deviation 0.025 0.413 11.615 49.983 


Table 3. FER and Rounds Fired Summary Statistics 
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IV. STATISTICAL ANALYSIS 


This chapter discusses the methodology for analyzing the results and the 
statistical tests used in the analysis. 
A. ANALYSIS METHODOLOGY 

After completing all runs and gathering required data, rigorous statistical tests 
were applied to determine whether HLA Warrior results were statistically similar to Janus 
results. For each simulation, the same random number seed was used in both Janus and 
the eoueeponuine HLA Warrior run. The only difference in the scenarios was the mode 
(Janus or HLA Warrior) on which it was executed. Since identical scenarios were 
executed in identical environments on opposing simulations, the data from the two 
simulations is paired. Given the data was paired between Janus and HLA Warrior, two 
applicable statistical tests were used to determine similarity: the Wilcoxon Signed Ranks 
Test and the paired t-test. Initially, all MOPs were analyzed using the Wilcoxon Signed 
Ranks test, then the MOPs that satisfied the paired t-test normality assumption were 
analyzed using the more powerful paired t-test. 
B. TOOLS FOR ANALYSIS 

1. Wilcoxon Signed Ranks Test 

As the actual distributions of the MOPs are not known, all MOPs were first 
analyzed using a pair-wise comparison by ies of the non-parametric Wilcoxon Signed | 
Ranks Test. The Wilcoxon Signed Ranks Test begins by taking the data consisting of n 
observed pairs (X), Y;), (X2, Y2), ..., (Xn Yn), and computing the absolute difference (ID,|) 


between each of the n pairs. 


i 








IDjl = lY;-X;| i= 1,2,....n 
Cases where the differences are zero (X; equals Y;) are omitted. The number of remaining 
pairs is then denoted by n’,n’<n. Ranks from 1 ton’ are then assigned to the n’ pairs 
according to the relative size of their absolute differences as follows. Rank 1 is given to 
the pair (X;, ¥;) with the smallest absolute difference IDjl; rank 2 is given to the pair with 
the second smallest difference, and so on. In cases of a tie, the average of the ranks that 
would have been otherwise assigned is assigned to each of the pairs in the tie. 
[Ref. 5:p. 206-207] 

First, the Wilcoxon Signed Ranks Test relies on several assumptions with regard 
to the D;’s. D;’s are assumed to be continuous and independent. These assumptions are 
justified by the construction of the D;’s; each D; is the difference of two real numbers and 
each originates as a result of independently selected random number seeds. Another 
assumption is the distribution of the D,’s is assumed to be symmetric. If the two 
simulations are the same (as they are under the null hypothesis that the HLA Warrior and 
Janus populations are identical) the average amount by which Janus MOPs exceeds HLA 
Warrior MOPs should be identical to the average amount by which HLA Warrior MOPs 
exceeds Janus MOPs. The final assumption is the mean of the distribution of the D,’s is 
some hypothesized value (given the null hypothesis, the value of the mean is zero). 

[Ref. 5:p. 207] 

The test statistic, T, equals the sum of the ranks (R;) assigned to those pairs where 

Y; exceeds Xj. 
R;=0 if X;>Y; (D; negative). 
R; = the rank assigned to (X;,Y;), if Xi<Y; (D; positive). 


16 








Therefore, the test statistic is given by: 
T = 2R;. 

For this thesis, the null hypothesis, Ho, is that the mean value of the population of 
X;'S (HLA Warrior results from all possible random number seeds) is equal to the value 
of the Y;’s (Janus results). The alternate hypothesis is that the HLA Warrior results and 
the Janus results are not equal. The null hypothesis is rejected in favor of the alternate 
hypothesis at the level of significance « = 0.05 if: 

T < Wo2 Or T > Wy-a2. 

Critical lower and upper values of w,, the p™ quantile [Ref.6:p.208], with a = 0.05 and 
n=10, are 9 and 46, respectively [Ref. 5:p. 383]. The p-values (the smallest level of 
significance at which the null hypothesis would be rejected when a specified test 
procedure is used [Ref. 4:p. 334)) for the test statistic for each MOP was then determined 
using the S-Plus statistical software package [Ref. 6:p. 82]. 

2. Paired t-Test 

The paired t-test relies on the assumption that both data sets to be compared come 
from normal distributions. Visual observation of normal probability plots of each of the 
MOPs was used to determine if the data was plausibly normal. For those cases 
determined to be normal, the paired ¢-test was applied. As discussed previously in 
chapter three, the only difference in each run was the mode in which the scenario was 
run; therefore the results of corresponding HLA Warrior and Janus runs are paired and 


utilization of the paired t-test is appropriate. 
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The paired t-test takes the data that consist of n observed pairs (X;,Y,), (Xz, Y2), ..., 

(X,,, Y,), and computes the difference (d;) between each of the n pairs. For this thesis, the 
X; s are HLA Warrior MOP results and the Y;’s are Janus MOP results. The mean of the 
n differences 1s then computed. Under the null hypothesis, the mean of the differences 
for each of the MOPs 1s zero in the population, and under the alternative hypothesis, the 
mean of the differences is not zero. 

Null Hypothesis: H,: Up = 0 

Alternative Hypothesis: H,: Up # 0 
The test statistic, t, for the analysis is defined as: 


d-A 
t= z 





where 





and the sample standard deviation is 


Sd? -(Sd,)" In 
Ae = as ae ‘ 


The rejection region for the null hypothesis is: 


C2b ayant OF ES—bg sont 
with confidence level a and n — 1 degrees of freedom. If t falls in the rejection region, 
the null hypothesis is rejected and the conclusion is that HLA Warrior and Janus are not 


statistically similar. [Ref. 4:p. 367-368] For this thesis, a= 0.05. 
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C. RESULTS 

1. Wilcoxon Signed Ranks Test 

The Wilcoxon Signed Ranks Test indicates that HLA Warrior and Janus produce 
similar results in only a few instances. Appendix B lists the results of the Wilcoxon 
Signed Ranks Test. Table 4 summarizes the results for each MOP. Only three of the 24 


possible outcomes showed that HLA Warrior and Janus are statistically similar. 


Measure Of Performance 


Blue Red Blue Red Force 
Detection | Detection Kill Kill Exchange; Rounds 
_Environment_| Scenario | Range Range Range Range Ratio ee 


Hunter Liggett Mech | Not Similar| Not Similar Fie Similar | Not Similar | = Sinii 


Table 4. Wilcoxon Signed Ranks Test Results 























a. Detection Range 

In all environments and scenarios, the null hypothesis that HLA Warrior 
and Janus produced similar results was rejected with p-values less then 0.00195 in every 
case. Interestingly, the results depended heavily on the scenario. In the light scenarios, 
HLA Warrior always produced detection ranges exceeding Janus detection ranges. 
Conversely, in the mechanized scenarios, Janus always produced detection ranges 


exceeding HLA Warrior detection ranges. 
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b. Kill Range 

In one of the eight combinations of scenario and environment, the test 
leads to a failure to reject the null hypothesis that the population mean kill ranges 
produced by HLA Warrior and Janus are identical. The blue kill ranges in the Hunter- 
Liggett, mechanized scenario were found to be statistically indistinguishable with a p- 
value of 0.375. In all other combinations the null hypothesis was rejected with p-values 
less then 0.0058 in each case. 

c. Force Exchange Ratio 

Only the Fort Hunter-Liggett scenarios led to a failure to reject the null 
hypothesis that the force exchange ratios produced by HLA Warrior and Janus are 
similar. The p-value for the light infantry scenario was 0.492 and the p-value for the 
mechanized scenario was 0.083. The null hypothesis was rejected for both scenarios, 
light infantry and mechanized, executed in Southwest Asia with p-values less than 0.002. 

d. Rounds Fired 

In all cases, the null hypothesis that HLA Warrior and Janus fire the same 
number of rounds was rejected with p-values of less than 0.00391 in each instance case. 
In all cases, the number of rounds fired by Janus was much greater than the number of 
rounds fired by HLA Warrior. In one instance, the Hunter-Liggett light scenario, Janus 
fired an order of magnitude greater than HLA Warrior. 

2. Paired t-Test 
After the analysis using the Wilcoxon Signed Ranks Test, the data was analyzed 

‘ies the paired t-test for those data sets satisfying conditions of normality. The first step 


of the paired f-test was to determine if the data followed a normal distribution using 
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normal probability plots. The results of the normality evaluations are found in Table 5 


below. The normal probability plots are shown in Appendix D. 


Measure Of Performance 


Detection | Detection Kill Kill Exchange | Rounds 
__Environment | Scenario| Mode _| Range Range Range Range Ratio Fired 
| Hunter-Liggett | Light | Janus | Normal | Normal _| Normal [NotNormal 
| Hunter-Liggett || Mech | Warrior [Normal _[NotNonmall Normal [Normal [NotNGrial] Normal | 
Hunter Ugget_|wesh|Janae|-Nemal_|-Woma!_["Nommal{ Nema nomal_[ Norma 
‘Not Normal 


Table 5. Normal Probability Plot Results 














Those MOPs found to be normally distributed were then subjected to the paired t- 
test. The paired t-test confirms the Wilcoxon Signed Ranks Test’s findings that HLA 
Warrior and Janus generally do not produce similar output. Appendix C contains the 


paired f-test results. Table 6 summarizes the results of the paired t-test. 


Measure Of Performance 






Rounds 











Fired 
Hunter-Liggett Not Similar Not Similar | Not Similar — ar] Not Similar 


Hunter-Liggett TWisch | Not Simiar) —-NA— FSi nilar| Not Similar] NA Not Similar 
Southwest Asia Not Similar Not Similar] Not Similar 
Southwest Asia Not Similar Not Similar Not Similar] Not Similar 


Table 6. Paired t-Test Results 
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a. Detection Range 

Similar to the Wilcoxon Signed Ranks Test, in all combinations of 
scenario and environment the null hypothesis that HLA Warrior detection range is 
statistically similar to Janus’ detection range was rejected. P-values were less than 
0.000567 for the detection range in every case. 

b. Kill Range 

The null hypothesis that the results are similar was rejected in all cases of 
kill range with the exception of the Hunter-Liggett mechanized scenario, which resulted 
in a p-value of 0.983. P-values for the cases where the null hypothesis was rejected 
ranged from 1.84E-06 to 0.00219. 

c. Force Exchange Ratio 

The results from the paired t-test for Force Exchange Ratio confirmed the 
results of the Wilcoxon Signed Ranks Test. In both scenarios executed in the Hunter- 
Liggett environment, the null hypothesis was not rejected with p-values of 0.529 and 
0.082 for the light and mechanized scenarios, respectively. However, the Hunter-Liggett 
mechanized scenario failed to pass the normality test and therefore the results are not 
reliable. The null hypothesis was rejected for both scenarios executed in Southwest Asia 
with p-values of 0.000153 and 3.88E-07 for the light and mechanized scenarios, 
respectively. 

d. Rounds Fired 

The paired t-test for the number of rounds fired also confirmed the 
Wilcoxon Signed Ranks Test. In all cases the null hypothesis that the number of rounds 
fired in both simulations was similar was rejected. The p-values ranged from 0.0118 for 
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the Hunter-Liggett mechanized scenario to 3.65E-12 for the Hunter-Liggett light 
scenario. 
D. ANALYSIS OF RESULTS 

The results of the Wilcoxon Signed Ranks Test and the paired t-test clearly 
indicate that HLA Warrior and Janus do not produce statistically similar results with 
respect to the tested MOPs. With few exceptions the null hypothesis that Janus MOPs 
are Statistically similar to HLA Warrior MOPs was rejected. The next step was to 
analyze why differences exist and identify potential causes. Two sources of the 
differences investigated were HLA Warrior source code errors and differences in the 
implementation of the Janus algorithms in HLA Warrior. 

1. Source Code Errors 

Given that HLA Warrior is a new simulation still undergoing verification and 
validation testing, a starting point for identifying causes to the disparate results was the 
source code logic. Analysis of the Bede began with the search and detection algorithms 
and kill algorithms. An error in the detection algorithm would likely lead to errors in the 
other MOPs. For instance, changes to detection range would likely affect the kill range 
since these MOPs are closely linked. Also, the number of rounds fired will impact the 
number of kills and potentially kill ranges. Further the number of kills will ultimately 
affect the force exchange ratio in the battle. 

Initial investigation into the search and detection algorithm and the kill algorithm 
found two errors in the HLA Warrior source code. The first error was in the inter-fire 
time calculations. In Janus, there are several parameters that can be specified for each 
weapon system. Among them are range, rounds per trigger pull, trigger pulls per reload, 


23 








and reload times. Recall all scenarios were developed in Janus then converted to HLA 
Warrior. As HLA Warrior read the Janus data, a conversion error occurred when 
processing the number of trigger pulls per reload. Specifically, HLA Warrior 
exaggerated the inter-fire times for systems with the trigger pulls per reload parameter 
exceeding one. Therefore the overall number of rounds fired by HLA Warrior was less 
than expected. 

The second error in the source code allowed the AH-64s in the mechanized 
scenarios to detect and engage targets behind them. In essence the AH-64s fired Hellfire 
missiles backwards, which is not possible. This was evident while watching the 
mechanized scenario executed in Southwest Asia. In each run, the red AH-64s over flew 
then systematically killed the blue tanks. As a result the blue force suffered almost total 
annihilation while inflicting almost no casualties on the red force. 

HLA Warrior developers corrected both errors and all of the Hunter-Liggett light 
infantry scenarios and Southwest Asia mechanized scenarios were re-executed. Analysis 
of the results showed that corrections implemented to HLA Warrior source code lessened 
the gap between Janus and HLA Warrior in most MOPs. In the Southwest Asia 
mechanized scenario, the p-value for number of rounds fired changed from 1.02E-07 
prior to the corrections to 0.018 afterward. Although the null hypothesis that HLA 
Warrior and Janus produce similar results was still rejected, the gap between them was 
significantly reduced. In the case of red kill ranges, prior to implementing the changes, 
the null hypothesis that the simulations were the same was rejected with a p-value of 
0.0013. After the changes, the null hypothesis is no longer rejected with a p-value of 
0.108. 
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In the Hunter-Liggett light infantry scenario, the disparity between Janus and 
HLA Warrior was reduced in four of the six MOPs, blue detection range, blue kill range, 
FER, and rounds fired. The largest improvement was in the number of rounds fired. 
While HLA Warrior still fires significantly less than J anus, the magnitude of the 
difference was reduced by a factor of 10. Tables 7 and 8 below summarize the results of 
the Wilcoxon Signed Ranks Test and the paired t-test for the Hunter-Liggett light infantry 
and the Southwest Asia mechanized scenarios following the corrections to HLA Warrior. 
Increased p-values indicate greater similarity between Janus and HLA Warrior results. P- 
values greater than 0.05 fail to reject the null hypothesis that the results are statistically 
similar. Appendix E contains the raw data and analysis output for the comparison of and 
the Hunter-Liggett light infantry and the Southwest Asia mechanized scenarios utilizing 


the corrected version of HLA Warrior. 


Measure of Performance 


Blue Red Blue Red Force 
Detection | Detection Kill ~ Kill Exchange! Rounds 
Range Range Range Range Ratio Fired 


|SWA Mech] Before | 0.000036 | 0.000001 [ 0.001033 | 0.001309 0.000000 | 0.000000 | 
|SWA Mech] After__| 0.000002 | 0.000067 | 0.013039 0.108290 0.000001 | 0.018440 | 


Table 7. Signed Ranks Test Results (p-Values) Before and After HLA Warrior Modifications 
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Measure of Performance 


Blue Red Biue Red Force 
Detection | Detection Kill Kill Exchange | Rounds 
Range Range Range Range Ratio Fired 


Table 8. Paired t-Test Results (p-Values) Before and After HLA Warrior Modifications 













: 


While the modifications reduced the disparity between Janus and HLA Warrior, 
differences still exist between the two simulations. In all of the 30 light infantry 
scenarios (including the 10 runs following the corrections to the programming), HLA 
Warrior entities recorded detection events at a greater distance than Janus entities. 
Conversely, in every instance in the mechanized scenarios, Janus detection ranges 
exceeded HLA Warrior detection ranges. As one might expect, kill ranges tended to 
follow the pattern observed in the detection ranges. In. the light scenarios, HLA Warrior 
tended to have longer kill ranges. Conversely, in the mechanized scenarios, J sails tended 
to have longer kill ranges. Therefore, further investigation into the programming may be 
warranted. 

2. Methodology Differences 

Investigation into the methodology of the programming of HLA Warrior led to 
several possibilities that could account for the differences between the two simulations. 
Among them are the way HLA Warrior reads terrain files and the way HLA Warrior 


implements some Janus algorithms. 


26 














a. Terrain Conversion 


As mentioned previously, HLA Warrior possesses the capability to read 
scenario files constructed in Janus. This thesis took advantage of the capability to ensure 
that all corresponding scenarios began in exactly the same situation. When reading entity 
locations from Janus, HLA Warrior converts all Universal Transverse Macerator (UTM) 
grid locations into Latitude-Longitude locations [Ref. 7]. Due to the inherent errors in 
the conversion process, when HLA Warrior positions entities, their locations may be 
displaced slightly when compared to their original locations in Janus. This displacement 
could ultimately effect the line of sight for those entities. In hilly or rough terrain similar 
to Fort Hunter-Liggett, entities that previously had clear fields of view in Janus may be 
placed behind terrain features once converted to HLA Warrior, inhibiting line of sight. 
Reducing the line of sight will adversely effect detection ranges. 

b. Event Scheduling 

While HLA Warrior developers took great efforts to ensure that the Janus 
algorithms remained intact, the algorithms were often implemented differently in HLA 
Warrior. The primary implementation difference is in the way events are scheduled prior 
to the execution of the individual algorithms. Janus has approximately ten major events 
processing at regular intervals. Among them are move events, search events, detection 
events, and impact events. As the simulation clock advances, each of the major events is 
processed in a predetermined priority. The search event is executed as follows. Prior to 
Starting the simulation run, the user can specify a variable called dtsearch. This variable 
represents the time required for an entity to conduct a search. For instance, let dtsearch 
equal 6. The simulation then divides dtsearch by the number of sides in the scenario (say 
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2). Each side in the scenario is then assigned an equal portion of dtsearch to conduct its 
searches. So, for example, each entity on side one conducts its first search at time zero. 
All entities on side one then conduct the next search dtsearch time units later, at time six 
in this example. The entities on side two conduct their first search at time three in this 
example. Their subsequent search will commence dtsearch time units later, at time nine. 
This search pattern continues throughout the duration of the simulation. Had there been 
three sides, the first side would have conducted searches at times zero, SIX, twelve, and so 
on; side two would have conducted searches at time two, eight, fourteen, and so on; and 
side three would have conduct searches at times four, ten, sixteen and so on. [Ref. 7] 

Each search event then stochastically determines if a future detection 
event will be scheduled through use of the line-of-sight and detection algorithms. 
Similarly, detections can schedule engagements, and subsequent kill events, all of which 
are executed as the simulation clock advances. [Ref. 7] 

HLA Warrior utilizes the same major events as Janus, but they are 
implemented differently. In the case of the search event, HLA Warrior also utilizes a 
variable called dtsearch, defined identically to the Janus variable. However, HLA 
Warrior applies a different method to determine when entities conduct searches. Every 
entity is assigned an initial search time according to a uniform random variable on the 
interval zero to dtsearch. Recall that in Janus, all entities on a side conduct searches at 
the same time. The second search for each HLA Warrior entity takes place dtsearch time 
units after the first search. As a result individual entities on both sides are conducting 


searches throughout the dtsearch time period. Consequently, searches by both sides are 


28 











ongoing continuously for the duration of the simulation. Once searches commence, the 
algorithm to determine detections is identical to the Janus algorithm. [Ref. 7] 

These scheduling difference may lead to differences in detections and 
provide a partial explanation as to why the detection ranges in the HLA Warrior are 
different from the detection ranges in Janus. Recall that in Janus, all detections for a 
particular side occur at the same time, then are repeated dtsearch time units later, where 
as in HLA Warrior, detection events occur continuously throughout the simulation. In 
the time between search events in Janus, all entities will have the opportunity to move 
greater distances than HLA Warrior entities. The differences in entity position between 
each detection event may account for the disparity in average detection range between 
| Janus and HLA Warrior. 

While the event scheduling in HLA Warrior is different from the 
implementation in Janus, HLA Warrior more closely models reality. In an actual combat 
environment, individual soldiers and weapons systems are not limited to conducting 
searches at specific times. Rather, they conduct searches throughout the course of the 
battle. Given that, HLA Warrior’s results may be more realistic than Janus’. 

c. Line-of-Sight Algorithm 

HLA Warrior’s line-of-sight algorithm has also been modified compared 
to Janus’ line-of-sight algorithm. The modifications were implemented in order to reduce 
the overall complexity, and thus execution time of the algorithm. In basic terms, when 
determining the line of sight between two entities, Janus first determines which opposing 
entities are within range of the sensor of the searching entity. For those entities within 
range, Janus then “walks the line” between the entities three times. The first time it 
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walks the line, Janus collects data on the terrain and features in the vicinity of the line. 
On the second walk Janus checks to determine if terrain interferes with the line of sight. 
The third walk then determines if any features in the vicinity of the line (trees, buildings, 
etc) interfere with the line of sight. These calculations are then computed for every entity 
combination on the battlefield. [Ref. 7] 

HLA Warrior, on the other hand, only walks the line of sight twice. On 
the first walk, HLA Warrior determines which features on the terrain are in the proximity 
of the line of sight and could possibly interfere with the line. On the second walk, both 
terrain and features are analyzed. Additionally, the HLA Warrior designers developed a 
proximity manager that reduces the number of entity pairs that have to be examined. The 
proximity manager is essentially a database that tracks the locations of all entities in the 
simulation. It is updated after the execution of each move event. When determining line 
of sight, the searching entity queries the proximity manager as to which entities of the 
opposing force are within its sensor search pattern, then only those entity pairs within the 
search pattern are investigated further for line of sight. (The proximity manager is also 
utilized in artillery casualty assessment, chemical casualty assessments, and movement 
delays caused by obstacles and terrain features). Once line of sight is determined, the 
detection and kill algorithms in HLA Warrior and Janus are identical. [Ref. 7] 

A probable difference in the outcome of the detection algorithm as a result 
of implementation differences is likely in the proximity manager. When determining line 
of sight, Janus looks at all terrain and features within the range of the sensor, while HLA 
Warrior looks at only those entities passed to the detection algorithm by the proximity 


manager. Future investigation into the implementation of the proximity manager is 
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required to determine if all applicable terrain features are being processed for line-of- 
sight determination, particularly with regard to mechanized and light infantry scenarios. 
The net result of these different implementations is that HLA Warrior 
tends to be more efficient in its calculations and the execution of the algorithms. 
However, the implementation of the algorithms may cause the disparate results in the two 


simulations. 
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V. CONCLUSIONS 


The initial comparison between HLA Warrior and Janus indicated that the 
simulations do not produce statistically similar results with respect to most of the selected 


MOPs. In almost all cases the null hypothesis that the simulations were the same was 





rejected. Following the initial analysis, two HLA Warrior programming errors, inter-fire 
times between reloads and helicopter search and fire sectors, were identified and 
corrected. The subsequent analysis revealed that, while the simulations were still 
statistically dissimilar, some differences between Janus and HLA Warrior were reduced. 
Specifically, differences in number of rounds fired and kill ranges were reduced and the 
FER was indistinguishable following the corrections. However, consistent differences 
remain in detection ranges between Janus and HLA Warrior. 

Certain areas in HLA Warrior warrant further investigation. First, modification of 
the HLA Warrior search sequence to match Janus’ search sequence and re-running the 
scenarios would help isolate whether HLA Warrior’s current search sequence is causing a 
disparity between the simulations. Modifying the search sequence would be a relatively 
Straightforward and inexpensive modification to HLA Warrior. Also, follow-on studies 
should focus on examining the algorithms in both simulations to determine if they are in 
fact identical. Finally, additional research is necessary to assess HLA Warrior’s line-of- 
sight algorithm. Specifically researchers should investigate the implementation of the 
proximity manager to ensure all applicable terrain features are being processed for line- 


of-sight calculations. 
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Janus is considered valid primarily through user confidence and face-validity. 
After observing 40 scenarios executed in Janus and the same scenarios executed 60 times 
in HLA Warrior, both Janus and HLA Warrior scenarios produce generally similar 
outcomes and the HLA Warrior results were plausible. Following the HLA Warrior 
source code corrections, the entities appeared to engage and kill opposing entities at 
believable distances and locations, and the flow of the battles occurred as one might 
expect. Furthermore, Janus and HLA Warrior did produce statistically similar force 
exchange ratios, the quantitative measure of the scenario outcome. Consequently, while 
HLA Warrior and Janus produce statistically different detection and kill ranges, scenario 


outcomes are similar and HLA Warrior demonstrates a degree of face-validity. 
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APPENDIX A —- RAW DATA 


SCENARIO #1 HL Light 


Detection Range Blue | Detection Range Red Kill Range Blue Kil! Range Red Rounds Fired 

















































































| Run sits Warrior Janus Warrior Janus Warrior Janus Warrior Janus Warrior — Janus Warrior Janus 


1 1297.10 1098.42} 1353.50 1168.03; 1049.25 883.86} 1200.88 1067.67 

2 1341.90 1113.54 1388.80 1146.07) 1060.44 870.63} 1114.91 1045.07 

3 1307.00 1080.47} 1359.20 1155.76) 1107.67 901.79} 1100.87 1033.92 

4 1296.00 1042.76) 1338.90 1174.89} 1037.43 970.36} 1130.08 1047.87 

5 1309.30 1063.36] 1375.90 1156.48[ 1032.46 940.58; 1117.25 1085.00 

6 1301.40 1106.17} 1326.80 1152.94 954.48 904.74; 1061.21 1062.74 

7 1317.20 1141.20} 1377.10 1165.32} 1052.65 853.86; 1157.30 1045.92 

8 1306.70 1081.64} 1356.30 1157.09 992.80 933.66]; 1421.31 1045.84 

9 1308.70 1029.20} 1376.70 1146.74, 1072.54 825.17; 1129.76 1013.87 

10 1287.00 1095.05} 1337.60 1156.57} 1010.95 832.76} 1134.82 998.27 
Average 1304.23 1082.18} 1359.08 1157.99) 1037.06 891.74; 1126.84 1044.61 0.187 0.171 6690.2 71883.1 
Variance 78.88 838.73 416.76 82.73} 1851.44 2271.77; 1304.39 634.77} 0.0009 0.0031} 180643.3 18628348.3 
Standard Deviation 8.88 28.96 20.41 9.10 43.03 47.66 36.12 25.191 0.0306 0.0559 425.0 4316.1 































SCENARIO #2 HL Mech 


Detection Range Blue | Detection Range Red Kill Range Blue KillRangeRed | FER | Rounds Fired 
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/ Run | Warrior Janus Warrior Janus Warrior Janus Warrior Janus Warrior —_ Janus Warrior Janus 


1 3546.19 4077.79; 3092.52 3243.25} 3008.90 2603.06] 2756.89 3099.05 

2 3532.47 3947.62); 3099.91 3227.77} 2869.33 2584.04) 2790.48 3028.95 

3 3553.42 4080.81] 3044.10 3339.80] 2791.31 2802.86] 2679.50 3205.00 

4 3482.95 4512.96} 3062.95 3686.48} 2830.76  3526.67| 2607.39 3475.63 

5 3534.32 4018.41] 3093.82 3242.85) 2901.20 2834.78] 2820.08 2883.33 

6 3612.19 3896.11} 3109.91 3387.61) 3096.67 3001.56; 2808.21 2857.86 

7 3548.88 4170.06} 3174.27 3766.03} 3155.58 3140.30; 2831.13 3295.88 

8 3444.83 3900.53) 2949.61 3402.75} 2832.18 2753.80; 2497.97 3076.25 

9 3545.60 3982.52] 3192.82 3479.09} 2939.94 2824.12! 2763.62 2956.15 

10 3477.94 4477.38) 2896.87 3677.49} 2799.30 3132.31] 2642.98 

Average 3527.88 4106.42) 3071.68 3445.31] 2922.52 2920.35) 2719.82 3153.43 0.793 1.082 323.80 511.50 
Variance 2258.05 49261.11] 8297.72  40165.09] 16074.85 82158.59) 11992.05 67025.93] 0.0205 0.2303] 1153.51 32754.28 
Standard Deviation 47.52 221.95 91.09 200.41 126.79 286.63 109.51 258.89} 0.1431 0.4799 33.96 180.98 



































RAW DATA 


SCENARIO #3 SWA Light 


Detection Range Biue | Detection Range Red Kill Range Biue KiliRangeRed | FER | Rounds Fired 
















BOODNDMN FWD 














Average 
Vanance 
Standard Deviation 





3792.37 
3709.24 
3608.05 
3924.61 
3533.10 
3424.19 
3235.43 
3534.35 
3202.67 
4239.12 
3620.31 
98800.28 
314.33 























4514.06 
4524.34 
4357.00 
4337.02 
4662.36 
4509.96 
4564.56 
4270.55 
4478.59 
4480.99 













13651.35 


4469.94} 2675.03 3721.51] 3448.49 3018.58) 2258.87 2854.76 
40574.55 52378.20) 84358.73 37852.22) 45086.55 82289.41 
116.84 201.43 228.86 290.45 194.56 212.34 286.86 





2543.64 
2603.43 
2652.03 
2858.22 
2849.94 
2801.35 
2356.39 
2566.15 
2500.31 
























3795.37 
4135.74 
3371.48 
3486.64 
3880.45 
3768.72 
3746.04 
3540.80 
3579.34 














| = Run ——sds Waarrior Janus Warrior Janus Warrior Janus Warrior Janus Warrior Janus Warrior Janus 


1 1481.00 1059.99 1822.90 1081.07 1200.82 619.70 

2 1572.20 1035.30] 1826.10 1106.32) 931.31 645.52 

3 1512.00 1135.79] 1803.40 1138.70! 1281.35 677.66 

4 1458.70 1089.99] 1757.00 1088.31] 723.36 704.71 

5 1415.80 1125.44 1714.00 1115.19] 812.65 684.79 

6 1477.30 1095.57] 1832.50  1116.47/ 985.64 621.91 

7 - 1534.60 995.47| 1803.10 1089.60] 1062.25 643.94 

8 1460.40 1051.76} 1796.30 1043.45] 974.01 671.62 

9 1426.40 1095.20] 1787.10 1100.69] 1107.16 693.43 

10 1468.90 1023.98} 1826.10 1074.15] 716.65 «617.92 

Average 1480.73 1070.85} 1796.85 1095.39] 979.52 658121 987.07 806.021 0.746 0.326] 18373.9  27723.3 

Variance 2273.85 2043.70] 1365.57 699.80] 36463.15 1050.69] 5016.07 1742.33] 0.0505 gan xori7 rrostoee 
Standard Deviation 47.68 45.21 36.95 san) 2e46018  os08 70.82 41.74} 0.2247 0.0886] 1518.9 4126.9 


3582.41 
3124.60 
3248.24 
2900.43 
3714.92 
3731.13 
3475.73 
3618.89 
3324.15 
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SCENARIO #4 SWA Mech 


Detection Range Blue | Detection Range Red Kill Range Blue KillRangeRed | FER | 
| Run | Warrior = Janus {| Warrior Janus__| Warrior Janus _{ Warrior Janus | Warrior Janus | Warrior Janus | 





Rounds Fired 





3098. 14 
3218.29 
2787.25 
2842.25 
3272.33 
3145.95 
3125.58 
2708.78 
2892.20 


2127.63 
2052.46 


2855.00 
3293.75 




















53 337 

0.128 1.770 46.70 91.90 
0.0006 0.1706 134.90 2498.32 
0.0255 0.4130 11.61 49.98 

















APPENDIX B 


el ee, 










HL Light 1 
HL Light 2 
HL Light 3 
HL Light 4 
HL Light 5 
HL Light 6 
HL Light 7 
i 8 

9 
10 












OCOOnN AG AGWDN = 


10 
















SWA Light |. 1 
SWA Light 2 
SWA Light 3 
SWA Light 4 
5 
6 
7 
8 
9 
10 





DOnNaAMA WN ~ 


10 









Average 

Variance 
Standard Deviation 
a 


Average 
Variance 
Standard Deviation 


=e 


Average 
Variance 
Standard Deviation 








Average 
Variance 
Standard Deviation 


Biue 


Det Range 


Janus 
1098.42 
1113.541 
1080.474 
1042.757 
1063.363 
1106.171 
1111,197 
1081.638 
1029.201 
1095.045 
1082.18 
838.727 
28.961 





Blue 


Det Range 


Janus 

4077.791 
3947.62 
4080,.812 
4512.963 
4018.409 
3896.107 
4170,064 
3900,532 
3982.519 
4477 379 
4106.42 


49261.111 


221.948 


Det Range 


Janus 
1059.985 
1035.304 
1135.793 
1089.992 
1125,443 


1095.567 
995.467 
1051.763 
1095.201 
1023.983 


rn nr i a pn TY 





1070.85 
2043.700 
45.207 


Det Range 


Janus 
4514.058 
4524.336 
4356.996 
4337.016 
4662.361 
4509.962 
4564.557 
4270.554 

4473.59 
4480.992 
4469.94 


13651.354 


116.839 









~GRMON DONO 


55 sum 
0 p-Value 
(9,46) Accept region 


ooooo0o 0 coco 


: 


of 


(9, a 


oooooo0o0ceoc 


-849.63 sum 
0.001 seu 25 p-Value 
(9,46) Accept region 










-578, 5 sum 
0.001 od) 25 p-Value 
(9,46) Accept region 






sum 
p-Value 
Accept region 
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Red 


Det Range 


Janus 
4168.028 
1146.065 
1155.756 
1174.892 
1156.483 

1152.94 
4165.323 
157.088 
1146.74 
1156.567 





1157,9883 


82.729 
9.096 


Red 


Det Range 


Janus 





3445. 311 
40165.087 
200.412 







Red 


Det Range 


Janus 
1081.068 
1106,317 
1138.703 
1088.305 
1115.187 
1116.474 
1089.595 
1043.445 
1100,.693 
826} 074.152 | 751.9 | DT 152 


~ | sum | 796.85 | 10953939 | 701.4561 | 55 | sum | 


699.803 
26.454 


Red 


Det Range 


Janus 
3795.37 
4135.74 

3371.475 
3486.64 
3880.451 
3768,7198 
3746.042 
3540.8 
3579.338 
3910.486 


3721.506 





52378.197 


228.863 


201.0917 55 sum 
0 p-Value 
(9,46) Accept region 


-373. 6328 ‘sum 
0. pejaesiae p-Value 
(9,46) Accept region 





701 “4561 sum 
. p-Value 
(9,46) Accept region 








-1046.4735 sum 
seeaocsiae p-Value 
{9,46} Accept region 





- WILCOXON SIGNED RANKS TEST RESULTS 


ee 









ooooo0o0 oa 000 












: 













oeoocoo0ococse 
* 

















WILCOXON SIGNED RANKS TEST RESULTS 


Biue Blue Red Red 
Kill Range | Kill Range Kill Range | Kill Range 
Warrior Janus Difference Warrior Janus Difference 























HL Light 1 9 
2 4 

3 3 

4 6 

§ 3 

6 0 

7 7 

8 5 

j 9 , : : : 8 

10 10 

Average 1037.06 891.74 145, 32 sum 1126.84 1044.61 55 sum 
- Variance 1851.44 2271.77 e 4 accept region 1304.39 634.77 accept region 
Standard Deviation 43.03 47.66 p-value 36.12 25.19 p-value 

























HL Mech 
HL Mech 
HL Mech 
HL Mech 
HL Mech 
HL Mech 
HL Mech 
HL Mech 
Hi Mech 
HL Mech 10 eg eee 


Average 2922.52] 2900.35) 217) 65 | sum | O79 BT SIS aa as TT «$5 “sum 
Variance 16074.85| 82158.59 (9, i accept region 11992.05] 67025.93 (9, a accept region 
Standard Deviation 126.79 286.63 p-value 109.51 258.89 D-value 










Omn nna wn — 






nee eae 
jlococcoccce 
































SWA Light 1 10 
SWA Light 2 5 
SWA Light 3 1 
SWA Light 4 2 
5 9 
6 7 
7 6 
8 3 
9 8 
10 ee ce) a, rn ee 











Average 979.52 658.12 55 sum 987.07 806.02, 181.05, 55 | sum 
Variance 36463.15 1050.69 (9, Pa accept region 5016.07 1742.33 (9, Me accept region 
Standard Deviation 190.95 32.41 p-value . 70.82 41,74 p-value 





Omonnonnh Wh = 





o : , : ; 
Average 3448.49 3018.58 429. 91 = sum 2258.87 2854.76 “598, 89 55 | sum | 
Variance 84358.73| 37852.22 (9, accept region| 45086.55| 82289.41 (9, ao accept region 
Standard Deviation 290.45 194.56 p-value 212.34 286.86 p-value 
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WILCOXON SIGNED RANKS TEST RESULTS 


ABS Signed 
Scenario haa Difference Dif Bent 









OLhUOAXONOWNM — 
ami 
ROOGZFOCOWNSO 





BLAND NAWN = 


















6 
Average 0. 18662 0. 1717] 0. 0156201 sum 34 
Variance 0.000936; 0.003121 P-Value 0.4921875 
Standard Deviation} 0.030601! 0.055867 accept region (9,46) 


SWA Light 
SWA Light 
SWA Light 
SWA Light 
SWA Light 
SWA Light 
SWA Light 
SWA Light 
SWA Light 
SWA Light 


Average 0. 746192 0. B26 0. 4201 917 sum 55 
Variance 0.050468} 0.007849 P-Value 0 
Standard Deviation] 0.224651] 0.088594 accept region (9,46) 











“-DRMHANONUYWS 











—~ On fN © M 












UNANOGAwWN- 


maooc°*coorhoo- 











Average OQ. 793652 1 “082 -0. 2893481 sum 10 
Variance 0.020465] 0.230262 P-Value 0.0839844 
Standard Deviation} 0.143056] 0.479856 accept region (9,46) 



























SWA Mech 1 2 

SWA Mech 2 9 ; 
SWA Mech 3 4 0 
SWA Mech 4 1 0 
SWA Mech 5 6 0 
SWA Mech 6 10 0 
SWA Mech ri 7 0 
SWA Mech 8 3 0 
SWA Mech 9 : 0 
SWA Mech 10 0 

Variance 0. 000648} 0. 170556 P-Value 0.0019531 
Standard Deviation| 0.025458] 0.412984 accept region (9,46) 
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WILCOXON SIGNED RANKS TEST RESULTS 


Rounds Rounds ABS Signed 
Warrior Janus Difference | Difference none 





1 7 

2 2 ; 
3 9 0 
4 3 0 
5 s) 0 
6 4 0 
7 10 0 
8 8 0 
9 1 0 
10 6 0 







Average 6690.20 71883.10| -65192.90 sum 0 
Variance 180643.29)] 18628348.32 accept region (9,46) 
Standard Deviation 425.02 4316.06 p-value 0.0039063 


SWA Light 
SWA Light 
SWA Light 
SWA Light 
SWA Light 
SWA Light 
SWA Light 
SWA Light 
SWA Light 
SWA Light 


Average 18373.90 27723.30 -9349.40 sum 0 
Variance 2307111.66) 17031082.46 accept region (9,46) 
Standard Deviation 1518.92 4126.87 p-value 0.0039063 
















=_é 
oO 



















OP ONAN AWN = 
Nn OOM Ww wo ~I 
COC 0OCOCCOC CO 














1 7 

2 10 : 
3 8 0 
4 4 0 
5 5 0 
6 1 0 
7 3 0 
8 9 0 
9 2 0 
10 6 0 












Average 323.80 511.50 -187.70 sum 0 
Variance 1153.51 32754.28 accept region (9,46) 
Standard Deviation 33.96 180.98 p-value 0.0039063 


















1 1 

2 6 ; 
3 3 0 
4 8 0 
5 4 0 
6 9 0 
7 5 0 
8 10 0 
9 2 0 
10 7 0 






Average 46.70 291.90 -245.20 sum 0 
Variance 134.90 2498.32 accept region (9,46) 
Standard Deviation 11.61 49.98 p-value 0.0039063 
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APPENDIX C - PAIRED f-TEST RESULTS 


Hunter Liggett Light Blue 
t-Test: Paired Two Sample for Means 
Warrior 
Mean 
Variance 
Observations 10 
Pearson Correlation 0.085404 
Hypothesized Mean Difference 0 
df 9 
t Stat 23.75604 
P(T<=t) one-tail 9.9E-10 
t Critical one-tail 1.833114 
P(T<=t) two-tail 1.98E-09 
t Critical two-tail 2.262159 


Hunter Liggett Mech Blue 
t-Test: Paired Two Sample for Means 


Warrior 

Mean 3527.878 
Variance 

Observations 10 
Pearson Correlation -0.429349 
Hypothesized Mean Difference 0 
df 9 
t Stat -7,433366 
P(T<=t) one-tail 1.98E-05 
t Critical one-tail 1.833114 
P(T <=t) two-tail 3.96E-05 
t Critical two-tail 2.262159 


Detection Range 


Janus 


1304.23 1082.181 
78.88456 838.7271 


10 


Janus 
4106.42 


2258.053 49261.11 


10 


4] 


Hunter Liggett Light Red 
t-Test: Paired Two Sample for Means 

Warrior 
Mean 
Variance 
Observations 10 
Pearson Correlation -0.387218 
Hypothesized Mean Difference 0 
df | 9 
t Stat 25.07204 
P(T<=t) one-tail 6.14E-10 
t Critical one-tail 1.833114 — 
P(T<=t) two-tail 1.23E-09 
t Critical two-tail 2.262159 


Hunter Liggett Mech Red 
t-Test: Paired Two Sample for Means 


Warrior 

Mean 

Variance 

Observations 10 
Pearson Correlation -0.089057 
Hypothesized Mean Difference 0 
df 9 
t Stat -5.19564 
P(T<=t) one-tail 0.000284 
t Critical one-tail 1.833114 
P(T<=t) two-tail 0.000567 
t Critical two-tail 2.262159 


Janus 


1359.08 1157.988 
416.764 82.72893 


10 


Janus 


3071.678 3445.311 
8297.719 40165.09 


10 








PAIRED t-TEST RESULTS 


Detection Range 


SWA Light Blue 
t-Test: Paired Two Sample for Means 
Warrior 

Mean 1480.73 
Variance 2273.847 
Observations 10 
Pearson Correlation -0.500798 
Hypothesized Mean Difference 0 
df 9 
t Stat 16.10573 
P(T<=t) one-tail 3.04E-08 
t Critical one-tail 1.833114 
P(T<=t) two-tail 6.07E-08 
t Critical two-tail 2.262159 


SWA Mech Blue 
t-Test: Paired Two Sample for Means 


Warnor 

Mean 

Variance 

Observations 10 
Pearson Correlation -0.203658 
Hypothesized Mean Difference 0 
df 9 
t Stat -7.527086 
P(T<=t) one-tail 1.79E-05 
t Critical one-tail 1.833114 
P(T<=t) two-tail 3.59E-05 
t Critical two-tail 2.262159 


Janus 
1070.85 


Janus 


3620.313 4469.942 
98800.28 13651.35 


10 
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SWA Light Red 
t-Test: Paired Two Sample for Means 

Warrior Janus 
Mean 1796.85 1095.394 
Variance 1365.569 699.8031 
Observations 10 10 
Pearson Correlation -0.120087 
Hypothesized Mean Difference 0 
df 9 
t Stat 46.25105 
P(T<=t) one-tail 2.58E-12 
t Critical one-tail 1.833114 
P(T<=t) two-tail 5.17E-12 
t Critical two-tail 2.262159 
SWA Mech Red 
t-Test: Paired Two Sample for Means 

Warrior Janus 
Mean 2675.033 3721.506 
Variance 40574.55 52378.2 
Observations 10 10 
Pearson Correlation 0.147282 
Hypothesized Mean Difference 0 
of 9 
t Stat -11.74603 
P(T<=t) one-tail 4.62E-07 
t Critical one-tail 1.833114 
P(T<=t) two-tail 9.24E-07 
t Critical two-tail 2.262159 





PAIRED t-TEST RESULTS 


Hunter Liggett Light Blue 
t-Test: Paired Two Sample for Means 


Warrior 
Mean 1037.063 
Variance 1851.439 
Observations 10 
Pearson Correlation -0.239252 
Hypothesized Mean Difference 0 
af 9 
t Stat 6.432146 
P(T<=t) one-tail 6.03E-05 
t Critical one-tail 1.833114 
P(T <=t) two-tail 0.000121 
t Critical two-tail 2.262159 


Hunter Liggett Mech Blue 
t-Test: Paired Two Sample for Means 


Warrior 
Mean 2922.519 
Variance 16074.85 
Observations 10 
Pearson Correlation 0.0145 
Hypothesized Mean Difference 0 
df 9 
t Stat 0.022014 
P(T<=t) one-tail 0.491459 
t Critical one-tail 1.833114 
P(T<=t) two-tail 0.982917 
t Critical two-tail 2.262159 


Kill Range 
Hunter Liggett Light Red 
t-Test: Paired Two Sample for Means 
Janus Warrior 
891.74 Mean 
2271.772 Variance 
10 Observations 10 
Pearson Correlation -0.001657 
Hypothesized Mean Difference 0 
df . 9 
t Stat 5.90019 
P(T<=t) one-tail 0.000115 
t Critical one-tail 1.833114 
P(T<=t) two-tail 0.000229 
t Critical two-tail 2.262159 
Hunter Liggett Mech Red 
t-Test: Paired Two Sample for Means 
Janus Warrior 
2920.349 Mean 
82158.59 Variance 
10 Observations 10 
Pearson Correlation -0.45706 
Hypothesized Mean Difference 0 
af 9 
t Stat -4.232935 
P(T<=t) one-tail 0.001099 
t Critical one-tail 1.833114 
P(T<=t) two-tail 0.002197 
t Critical two-tail 2.262159 
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Janus 


1126.839 1044.613 
1304.389 634.7711 


10 


Janus 


2719.824 3153.435 
11992.05 67025.93 


10 





PAIRED t-TEST RESULTS 





Kill Range 

SWA Light Blue SWA Light Red 
t-Test: Paired Two Sample for Means t-Test: Paired Two Sample for Means 

Warrior Janus Warrior 
Mean 979.5198 658.1215 Mean 987.0693 806.0225 
Variance 36463.15 1050.695 Variance 
Observations 10 10 Observations 10 
Pearson Correlation -0.085774 Pearson Correlation 0.670276 
Hypothesized Mean Difference 0 Hypothesized Mean Difference 0 
of 9 df 9 
t Stat - 5.17472 t Stat 10.82863 
P(T<=t) one-tail 0.000292 P(T<=t) one-tail 9.19E-07 
t Critical one-tail 1.833114 t Critical one-tail 1.833114 
P(T <=t) two-tail 0.000583 P(T<=t) two-tail 1.84E-06 
t Critical two-tail 2.262159 t Critical two-tail 2.262159 
SWA Mech Blue SWA Mech Red 
t-Test: Paired Two Sample for Means t-Test: Paired Two Sample for Means 

Warrior Janus Warrior 
Mean 3448.491 3018.577 Mean 2258.87 
Variance 84358.73 37852.22 Variance 
Observations 10 10 Observations 10 
Pearson Correlation 0.358888 Pearson Correlation -0.337755 
Hypothesized Mean Difference 0 Hypothesized Mean Difference 0 
df 9 df 9 
t Stat 4.757753 t Stat -4.590263 
P(T<=t) one-tail 0.000516 P(T<=t) one-tail 0.000654 
t Critical one-tail 1.833114 t Critical one-tail 1.833114 
P(T<=t) two-tail 0.001033 P(T<=t) two-tail 0.001309 
t Critical two-tail 2.262159 t Critical two-tail 2.262159 





45086.55 82289.41 


Janus 


5016.073 1742.328 


10 


Janus 
2854.76 


10 














PAIRED t-TEST RESULTS 


Force Exchange Ratio 


Hunter Liggett Light 
t-Test: Paired Two Sample for Means 
Warrior 

Mean 0.18662 
Variance 0.000936 
Observations 10 
Pearson Correlation -0.478422 
Hypothesized Mean Difference 0 
df 9 
t Stat 0.654641 
P(T<=t) one-tail 0.264536 
t Critical one-tail 1.833114 
P(T<=t) two-tail 0.529072 
t Critical two-tail 2.262159 


SWA Light 
t-Test: Paired Two Sample for Means 
Warrior 

Mean 0.746192 
Variance 0.050468 
Observations 10 
Pearson Correlation 0.32288 
Hypothesized Mean Difference 0 
df 9 
t Stat 6.23177 
P(T<=t) one-tail 7.65E-05 
t Critical one-tail 1.833114 
P(T<=t) two-tail 0.000153 
t Critical two-tail 2.262159 


Janus 
0.171 
0.003121 
10 


Janus 
0.326 
0.007849 
10 
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Hunter Liggett Mech 
t-Test: Paired Two Sample for Means 
Warrior 

Mean 0.792652 
Variance 0.020465 
Observations 10 
Pearson Correlation 0.234107 
Hypothesized Mean Difference 0 
df 9 
t Stat -1.957084 
P(T<s=t) one-tail 0.041014 
t Critical one-tail 1.833114 
P(T<=t) two-tail 0.082028 
t Critical two-tail 2.262159 


SWA Mech 
t-Test: Paired Two Sample for Means 


Warrior 
Mean 0.127833 
Variance 0.000648 
Observations 10 
Pearson Correlation 0.553716 
Hypothesized Mean Difference 0 
df 9 
t Stat -13.00034 
P(T<=t) one-tail 1.94E-07 
t Critical one-tail 1.833114 
P(T<=t) two-tail 3.88E-07 
t Critical two-tail 2.262159 


Janus 


Janus 


PAIRED t-TEST RESULTS 
Rounds Fired 


Hunter Liggett Light 
t-Test: Paired Two Sample for Means 


Warnor 
Mean 6690.2 
Variance 180643.3 
Observations 10 
Pearson Correlation 0.11606 
Hypothesized Mean Difference 0 
of 9 
t Stat -48.08282 
P(T<=t) one-tail 1.82E-12 
t Critical one-tail 1.833114 
P(T<=t) two-tail 3.65E-12 
t Critical two-tail 2.262159 


SWA Light 
t-Test: Paired Two Sample for Means 
Warnor 

Mean 18373.9 
Variance 2307112 
Observations 10 
Pearson Correlation 0.409474 
Hypothesized Mean Difference 0 
df 9 
t Stat -7.844541 
P(T<=t) one-tail 1.29E-05 
t Critical one-tail 1.833114 
P(T<=t) two-tail 2.59E-05 
t Critical two-tail 2.262159 


Janus 
71883.1 
18628348 
10 


Janus 
27723.3 
17031082 
10 
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Hunter Liggett Mech 
t-Test: Paired Two Sample for Means 





: Warrior 
Mean 323.8 
Variance 4153.511 
Observations 10 
Pearson Correlation -0.139225 
Hypothesized Mean Difference 0 
of 9 
t Stat -3.145005 
P(T<=t) one-tail 0.005916 
t Critical one-tail 1.833114 
P(T<=t) two-tail 0.011832 
t Critical two-tail 2.262159 


SWA Mech 
t-Test: Paired Two Sample for Means 


Warrior, 
Mean 46.7 
Variance 134.9 
Observations 10 
Pearson Correlation 0.019656 
Hypothesized Mean Difference 0 
of 9 
t Stat -15.17634 
P(T<=t) one-tail 5.1E-08 
t Critical one-tail 1.833114 
P(T<=t) two-tail 1.02E-07 
t Critical two-tail 2.262159 


Janus 
511.5 
32754.28 
10 


Janus 
291.9 
2498.322 
10 
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APPENDIX D —- NORMAL PROBABILITY PLOTS 


HL-Light Blue Detection Range HL Light- Blue Detection Range 


Janus 


“t1 -0.6 20,1 0.4 0.9 1.4 71.6 “1,1 0.6 «0.1 0.4 0.9 
Normat Dustribution Normal Dis tribution 


HL Mech Blue Detection Ranges 
HL Mech Blue Detection Ranges Janus 


Janus 


+41 -0.6 0,1 O.4 o> 1.4 “1.6 “1.8 -0.6 0.1 0.4 0.9 
Normal Otebutron Norm al Distribution 


SWA Light Blue Detection Ranges SWA Light Blue Detection Ranges 


HLA Warror Janus 


-t,1 0.6 “0.1 a 0.9 1.4 «1.6 Tt 0.6 0.1 0.4 0.9 
Normal Qistribulien Narm al Distribution 
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NORMAL PROBABILITY PLOTS 








SWA Mech Blue Detaction Ranges SWA Mech Blue Detection Ranges 
MLA Warrer Janu 
4300 
a10O 4600 
3900 
4400 
e 5 
5 3700 5 
3500 4400 
3300 
4300 
3100 
41,6 4.1 -0.6 Ot 0.4 0.9 14 -1.6 11 O06 Of 6.4 0,3 1.4 
Norm al Oratribution Normal Datrduton 
HL-Light Red Detection Ranges HL-Light Red Detection Ranges 
MLA -W aera Janue 
1300 
1t?2 
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F 3 1162 
= 3 
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1t$2 
1330 
1142 
“1.6 “2,1 0.6 0.1 O4 0.9 3,4 “1,6 -tt . “O86 -0.1 o4 0.2 1,4 
Normal Des trbution Norm a! Omtr ution 
HL Mech Red Detection Ranges Ht Mech Red Detection Ranges 
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3600 
3050 3 3$00 
= s 
3400 
2950 
3300 
3200 
2050 
91.6 ott 0.6 O14 0.4 0.9 t.@ -1.6 “V4 08 O.t 04 0.9 1.4 
Norm al Orstrdunon Norm a! Date @ut ion 
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NORMAL PROBABILITY PLOTS 


SWA Light Red Detection Ranges 





HLA Warrior 
1820 
g 1780 
= 
1740 
t700 
“1,6 “ET “0.6 0.1 0.4 0.9 t.4 
Normal Distribution 
SWA Mech Red Detection Ranges 
HLA Warrior 
3000 
2800 
£ 
= 
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SWA Light Red Detection Ranges 


Janus 
1120 
« 
a 
c 
s 
1080 
10480 
“1.6 V1 0,6 0.4 0.4 Coo 
Normal Datridution 
SWA Mech Red Detection Ranges 
Janus 
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NORMAL PROBABILITY PLOTS 
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NORMAL PROBABILITY PLOTS 


HL-Light Red Kill Ranges HL-Light Red Kill Ranges 


HLA-W arnor Janus 





; : 
> 7 
-1,6 *t. “0.6 -0.1 0.4 6.9 1.4 
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NORMAL PROBABILITY PLOTS 
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NORMAL PROBABILITY PLOTS 


SWA Light FER SWA Light FER 
BLA W arnor Janus 
1,2 
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NORMAL PROBABILITY PLOTS 


HL Mech Rounds Fired HL Mech Rounds Fired 
MLA Warroe Janus 
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APPENDIX E —- RAW DATA AND ANALYSIS RESULTS 
FOLLOWING HLA WARRIOR SOURCE CODE CORRECTIONS 


Southwest Asia Mechanized 
(Wilcoxon Signed Ranks Test Results) 










oie petectien Ran ge Red Detection Range 





HLA Warrior 





BSOMNAKHAwWH AIS 


ooo o cco °o 





oooo 0 coo © 





Average 3003, 34| 4469. Od -576. 60 Sum 3230, 70| 3721 x 490, 81 Sum 
Variance 12193.79] 13651.35 (9, 2 Accept Region 9855.70| 52378.20 (9, aA Accept Region 
Standard Deviation 110.43 116.84 0.001953 P-Value 99.28 228.86 0.001953 P-Value 


Blue Kill Ron ges Red Kill Ranges 


HLA Warrior] Janus [difference] rank | 























SODNOOH AON =I 





Glow ZS nmowan-sn 





oOorooo-+o on 








Average 3319. 22 3018. 38 300, 84 Sum 3675, 35 DB54, 76) -1 75. 41 Sum 
Variance 49370.27| 37852.22 ny - Accept Region 90462.01{ 82289.41 i546 Accept Region 
Standard Deviation 222.19 194.56 0.009766 P-Value 300.77 286.86 0.083984 P-Value 


















FER Rounds Fired 





HLA Warrior 





= 0 3 
2 6 
3 0 9 
4 0 1 
5 0 8 
6 0 4 
7 0 7 
8 0 10 
9 0 5 
10 . 2 








Average 0.283 1.770 -1 “487 Sum 590 292 55 Sum 
Variance 0.008 0.171 (9 a6 Accept Region 121 Se — (9, i Accept Region 
Standard Deviation 0.091 0.413 0.001953 P-Value P-Value 
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RAW DATA AND ANALYSIS RESULTS FOLLOWING HLA 


WARRIOR SOURCE CODE CORRECTIONS 


t-Test: Paired Two Sample for Means 


Blue Detection Range After Warrior Modifications 


Southwest Asia Mechanized 
(Paired t-Test Results) 


t-Test: Paired Two Sample for Means 


HLA Warrio Janus 


Mean 3893.34 4469.942 Mean 

Variance 12193.79 13651.35 Variance 
Observations 10 10 Observations 
Pearson Correlation -0.119579 Pearson Correlation 
Hypothesized Mean Difference 0 Hypothesized Mean Difference 
df 9 df 

t Stat -10.72004 t Stat 

P(T<=t) one-tail 1E-06 P(T<=t) one-tail 

t Critical one-tail 1.833114 t Critical one-tail 
P(T<=t) two-tail 2E-06 P(T<st) two-tail 

t Critical two-tail 2.262159 t Critical two-tail 


t-Test: Paired Two Sample for Means 


Blue Kill Range After Warrior Modifications 


3230.7 
9855.698 
10 
0.2714 

0 

9 


-6.947913 


3.35E-05 
1.833114 

6.7E-05 
2.262159 


t-Test: Paired Two Sample for Means 


HLA Warrio Janus 


Mean 3319.42 3018.577 Mean 

Variance 49370.27 37852.22 Variance 
Observations 10 10 Observations 
Pearson Correlation -0.091261 Pearson Correlation 
Hypothesized Mean Difference 0 Hypothesized Mean Difference 
cf 9 of 

t Stat 3.084757 t Stat 

P(T<=t) one-tail 0.00652 P(T<=t) one-tail 

t Critical one-tail 1.833114 t Critical one-tail 
P(T<=t) two-tail 0.013039 P(T<st) two-tail 

t Critical two-tail 2.262159 t Critical two-tail 


Red Kill Range After Warrior Modifications 
HLA Warrio Janus 


2675.35 
90462.01 
10 
0.414261 
0 

9 


-1.782835 


0.054145 
1.833114 

0.10829 
2.262159 


t-Test: Paired Two Sample for Means 
FER After Warrior Modifications 


HLA Warrio Janus 


Mean 0.2833 
Variance 0.008266 
Observations 10 
Pearson Correlation 0.340082 
Hypothesized Mean Difference 0 
df 9 
t Stat -12.00814 
P(T<=t) one-tail 3.83E-07 
t Critical one-tail 1.833114 
P(T<=t) two-tail 7.66E-07 
t Critical two-tail 2.262159 


1.77 
0.170556 
10 
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t-Test: Paired Two Sample for Means 
Rounds Fired After Warrior Modifications 


Red Detection Range After Warrior Modifications 
HLA Warrio. Janus 


3721.506 
52378.2 
10 


2854.76 
82289.41 
10 


HLA Warrio Janus 


Mean 590.1 
Variance 121950.3 
Observations 10 
Pearson Correlation 0.474476 
Hypothesized Mean Difference 0 
of 9 
t Stat 2.870962 
P(T<=t) one-tail 0.009224 
t Critical one-tail 1.833114 
P(T<=t) two-tail 0.018448 
t Critical two-tail 2.262159 


291.9 
2498.322 
10 


| 
| 





RAW DATA AND ANALYSIS RESULTS FOLLOWING HLA 
WARRIOR SOURCE CODE CORRECTIONS 


Hunter-Liggett Light Infantry 
(Wilcoxon Signed Ranks Test Results) 









Red Detection Range 




















1395.72 
1367.49 
1379.42 
1346.00 
1406.23 
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10° : : : 5 
Average 1313.32} 1082.18 55 Sum 1385.08 1157.99 = Sum 
Variance 236.29 838.73 (9, ns Accept Region 437.26 82.73 (9, i Accept Region 
Standard Deviation 15.37 28.96 P-Value 20.91 9.10 P-Vaiue 


Blue Kill Rang es pes Kill Rane es 


TA amor ars Siference} rank 


5 
8 
4 
0 
6 
3 
7 
: 
9 


ic : : 
Average 7063. 61 891. 74 sum 1142.38 1044.61 55 Sum 
Variance 5472.45} 2271.77 a 46) | Accept Region 418.92 634.77 (9, Mi Accept Region 
Standard Deviation 73.98 47.66 0.00195 P-Value 20.47 25.19 P-Value 


Rounds Fired 


HLA Warrior 






















OANA AGH = 





SHNWantpar ODN 



















-54995.00 
-60994.00 
-62142.00 
-53437.00 
-56036.00 


‘i ; : 
Average 0.19 0.17 Sum 15840. 20 71 B83. 10 Sum 
Variance 0.01 0.00 i 7” Accept Region} 7943005.29} 18628348.32 (9, on Accept Region 
Standard Deviation 0.09 0.06 0.5 P-Value 2818.33 4316.06 0.00195 P-Value 
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RAW DATA AND ANALYSIS RESULTS FOLLOWING HLA 
WARRIOR SOURCE CODE CORRECTIONS 


Hunter-Liggett Light Infantry 
(Paired t-Test Results) 


Hunter-Liggett Light Blue Detection Range 
t-Test: Paired Two Sample for Means 


HLA Warrior 
Mean 1313.321 
Variance 236.2907433 
Observations 10 
Pearson Correlation 0.026785455 
Hypothesized Mean Difference 0 
df 9 
t Stat 22.54443499 
P(T<=t) one-tail 1.57506E-09 
t Critical one-tail 1.833113856 
P(T<=t) two-tail 3.15012E-09 
t Critical two-tail 2.262158887 


Hunter-Liggett Light Blue Kill Range 
t-Test: Paired Two Sample for Means 


HLA Warrior 
Mean 1063.606 
Variance 5472.449471 
Observations 10 
Pearson Correlation -0.62083907 
Hypothesized Mean Difference 0 
df 9 
t Stat 4.936256062 
P(T<=t) one-tail 0.000403188 
t Critical one-tail 1.833113856 
P(T<st) two-tail 0.000806376 
t Critical two-tail 2.262158887 


Hunter-Liggett Light FER 
t-Test: Paired Two Sample for Means 


HLA Warrior 
Mean 0.1908 
Variance 0.007399956 
Observations 10 
Pearson Correlation -0.17982756 
Hypothesized Mean Difference 0 
df 9 
t Stat 0.565725575 
P(T<=t) one-tail 0.292703215 
t Critical one-tail 1.833113856 
P(T<=t)} two-tail 0.58540643 
t Critica! two-tail 2.262158887 


Janus 
1082.181 
838.7271 

10 


Janus 


Janus 
0.171 
0.003121 
10 
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Hunter-Liggett Light Red Detection Range 
t-Test: Paired Two Sample for Means 


Janus 





HLA Warrior 
Mean 
Variance 
Observations 10 
Pearson Correlation -0.28114945 
Hypothesized Mean Difference 0 
df 9 
t Stat 28.68035701 
P(T<=t) one-tail 1.85458E-10 
t Critical one-tail 1.833113856 
P(T<=t) two-tail 3.70915E-10 
t Critical two-tail 2.262158887 


Hunter-Liggett Light Red Kill Range 
t-Test: Paired Two Sample for Means 


HLA Warrior 
Mean 
Variance 
Observations 10 
Pearson Correlation -0.07247826 
Hypothesized Mean Difference 0 
df 9 
t Stat 9.203287939 
P(T<=t) one-tail 3.5548E-06 
t Critical one-tail 1.833113856 
P(T<=t) two-tail 7.10959E-06 
t Critical two-tail 2.262158887 


Hunter-Liggett Light Rounds Fired 
t-Test: Paired Two Sample for Means 


HLA Warrior 
Mean 15840.2 
Variance 
Observations 10 
Pearson Correlation -0.12546122 
Hypothesized Mean Difference 0 
df 9 
t Stat -32.5612979 
P(T<=t) one-tail 5.97732E-11 
t Critical one-tail 1.8331 13856 
P(T<=t) two-tail 1.19546E-10 
t Critical two-tail 2.262158887 


1385.077 1157.988 
437.2590456 82.72893 


10 


Janus 


1142.378 1044.613 
418.9244178 634.7711 


10 


Janus 
71883.1 


7943005.289 18628348 
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